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ABSTRACT 

The Metropolitan Educational Development and fiesearch 
Project (MEDARP) Docuaentation unit, located in the office of 
Educational Evaluation (OEE) at the New fork City Board of Education, 
is funded by the Ford Foundation to document, evaluate and provide 
technical assistance to the School Improvement and Local School 
Development Projects; these are major school-based constituency 
planning programs presently operating in the New York City public 
schools. The Onit also analyzes various documentary and evaluative 
data collected, and disseminates the results and conclusions of these 
analyses. The documentation function of the Onit involves the 
establishment and maintenance of an independent data base which will 
insure an impartial, detailed, continuous account of the development 
and progress of the School improvement and Local School Development 
Projects. The evaluation activities of the Onit emphasize ongoing 
project assessments which provide project staff, school participants, 
and program sponsors with useful feedback regarding the effectiveness 
of the projects* Technical assistance activities offered ro the 
projects include the development of documentation, evaluation and 
needs assessment methodologies* The findings of the evaluation 
reports and analytic papers prepared by the Onit will be widely 
disseminated through a variety of written and oral presentations. 
(Author/GK) 
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PROJECT ABSTRACT 



The MEDARP Documentation Unit is located in the Office of Educational 
Evaluation (GEE) at the New York City Board of Education. The Unit is 
funded by the Ford Foundation to document, evaluate and provide technical 
assistance to the School Improvement and Local School Development Projects; 
these are major school -based constituency planning programs presently 
operating in the New York City public schools. The Unit is also responsi- 
ble for the analysis of the various documentary and evaluative data, 
collected, as well as the dissemination of the results and conclusions of 
these analyses. 

The documentation function of the Unit involves the establishment and 
maintenance of an independent data base which will insure an impartial, 
detailed, continuous account of the development and progress of the School 
Improvement and Local School Development Projects. The evaluation activi- 
ties of the Unit emphasize ongoing project assessments which provide 
project staff, school participants, and program sponsors with useful 
feedback regarding the effectiveness of the projects. Technical assis- 
tance activities offered to the projects include the development of 
documentation, evaluation and needs assessment methodologies. The 
findings of the evaluation reports and analytic papers prepared by the 
Unit will be widely disseminated through a variety of written and oral 
presentations. 



INTRODUCTION 

The Documentation Unit to the School Improvement and Local School 
Development Projects is funded by a two-year grant from the Ford Founda- 
tion. In funding the Unit, the Ford Foundation hoped to ensure the 
maintenance of an ongoing record and the in-depth analysis of the 
process of school self-improvement as fostered by the New York Urban 
Coalition's Local School Development Project and the School Improvement 
Project of the New York City Board of Education. The Board of Education 
accepted the two-year foundation grant with the understanding that MEDARP 
would utilize the resources of its independent Steering Committee to over- 
see a neutral assessment of the two projects. Since project directors are 
normally too involved in carrying'out required activities, it is important 
that an outside unit describe project activities, record the progress of 
each activity, provide feedback to project personnel, and document the 
evolution and performance of the improvement projects. 

The Documentation Unit is a component of the Metropolitan Educational 
Development and Research Project (MEDARP) in the Office of Educational 
Evaluation. MEDARP is a planning project funded by the Spencer Foundation 
and the National Institute of Education (NIE). It was initiated to 
establish a research and development capacity for the public school system. 
The Documentation Unit, a.; a component of MEDARP, represents an effort by 
the Board of Education to provide educational programs, such as the School 
Improvement and Local School Development Projects, with research and 
evaluation capacities which are useful, relevant and directly supportive 
of ongoing program functions. 



A concern of the Ford Foundation, related to Documentation Unit data 
analysis activities, was the provision of a formal mechanism which would 
ensure the objectivity of the Unit. A subcommittee of the MEDARP Steering 
Committee, an independent advisory group made up of representatives of 
interested education organizations and institutions, has been formed to 
perform this role. The subcommittee has reviewed all documentation and 
assessment methodologies, instruments, analyses and reports developed by 
the Unit. Members of the Documentation Unit subcommittee include repre- 
sentatives of the United Parents 1 Association , the Community Superintendents 1 
Association, the State Educacion Department, and the Division of Curriculum 
and Instruction (BOE). 

In the remainder of this section the School Improvement and Local 
School Development Projects, and the functions and staffing of the 
Documentation Unit will be described. 

The School Improvement Project 

The School Improvement Project is a major innovative program developed 
and implemented by the central administration of the New York City Board of 
Education. Based on five factors identified by Edmonds (1979)^ as character- 
izing more effective schools, and the concept of local school site planning, 
the project's goal is to improve the educational achievement and environ- 
ment of New York City's public sthools. The five school effectiveness 
factors are: a) the administrative and i^tructional leadership of the 
principal; b) instructional emphasis on basic skills; c) a school climate 



^Ronald Edmonds, ''Effective Schools for the Urban Poor/ Educationa l 
Leadership, October, 1979, pp 15-27. " 



which is conducive to pupil learning; d) ongoing assessment of pupil 
progress; and e) optimistic teacher expectations of pupil ability. 
Through the efforts of a school-based planning committee representing 
the various school constituencies, and functioning in support of the 
principal, the conditions and practices contributing to a more 
effective school are to be established at each participating school. 

In 1979-80, nine public elementary schools and three non-public 
schools participated in the project, A school liaison is assigned to 
each school by the project (due to Federal funding regulations, only 
one liaison is working with the non-public schools). During the first 
year of the school's participation in the project, the liaison is 
responsible for conducting a needs assessment and assisting the 
principal in establishing the School Improvement Committee. A written 
improvement plan is prepared by the committee to address the needs 
identified in the school assessment and other concerns raised by the 
committee. 

Over the second and third years of project participation, the 
principal and committee implement the activities outlined in the improve- 
ment plan. The liaison's role at this point is to facilitate the delivery 
of available resources and materials requested in the plan. In addition, 
the liaison assists the committee in documenting and assessing the success 
of the implementation of the plan. Periodically the plan is revised to 
reflect the results of these interim evaluations. 



During the 1980-81 school year it is anticipated that 16 public and four 
non-public schools will be participating in the project. 



The Local School Development Project 

The Local School Development Project of the New York Urban Coalition 
is similar in design to the School Improvement Project in that Planning 
Teams, representative of the various school constituencies, are established 
at participating schools. The Planning Team is responsible for identify- 
ing the needs of the school and developing a Comprehensive School Plan to 
address these needs. 

A District Consultant, District Coordinator, and a Resource Coordina- 
tor are assigned as a District Team by the project to work with participa- 
ting schools in each of the five Community School Districts involved in 
the project. The duties of the five District Teams include the training 
and assistance of planning team members in needs assessment techniques, 
team building and comprehensive planning. The District Teams are also 
responsible for the identification and delivery of resources, services 
and materials to accomplish the goals and objectives of each school's 
comprehensive plan. 

In addition to the activities of the District Team, the Local School 
Development Project has developed a principal's training program and the 
Superintendents' Forum to involve Community Superintendents in the compre- 
hensive planning process. Thirty-seven public elementary and intermediate 
schools participated in the project during the 1979-80 school year (it is 
expected that these schools will continue to participate in the coming 
year). 

While both projects report to the Senior Assistant to the Chancellor 
for Instruction (the Local School Development Project is jointly managed 
by the New York Urban Coal it; on and the New York City Board of Education), 



each of the projects is also guided by an independent advisory council. 
These councils are composed of representatives cf public interest groups, 
the City and State education agencies and professional associations. The 
councils function in an advisory capacity, reviewing project policy and 
activities. 

The Docum e ntation Unit 

F unctions 

The Documentation Unit has four functions: documentation, formative 
evaluation, technical assistance, and information analysis and dissemination. 

Documentation . The documentation function of the Unit involves 
establishing and maintaining an independent data base to insure impartial, 
detailed, continuous records of the development and progress of the School 
Improvement and Local School Development Projects, Besides providing an 
ongoing account of the evolution and development of the two programs, the 
data base will provide the projects with 1) a reference source for retro- 
spective examination of specific project activities and events, 2) useful 
material for the purposes of training and briefing new program staff, and 
3) information useful in preparing project funding proposals. 

A long range goal of both projects is the development of replicable 
models of school improvement and institutional change. While the basic 
vehicle for promoting change is similar, i e., school site planning by 
constituents, project approaches are different in terms of program design. 
The Local School Development Project establishes a District Team responsible 
for project activities in several district schools and the local District 
Office. The School Improvement Project assigns one staff member to 
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each participating school. The longitudinal record of the development of 
these programs maintained by the Documentation Unit will serve as a 
reference source to assist policy-makers at the city, district and school 
levels in making decisions about tne adoption or development of alternative 
school-based planning models. 

Formative evaluation . Both projects are expected to evolve through 
implementation; therefore, a great deal of trial and error will be likely. 
To maximize the benefits of this exploratory approach to program develop- 
ment, the Documentation Unit will conduct a series of formative evaluations 
of the projects. The information collected will be available tc project 
administrators to guide them in their decisions to modify various project 
activities and program design components. 

The Unit will also provide participating schools with data which can 
be used to assess the success of school plan activities implemented during 
the year, and to prepare annual evaluation reports for project sponsors and 
funding sources (the Board of Education, the New York Urban Coalition, and 
the State Education Department). This evaluative data can also be used to 
identify the areas of school need to be addressed in the coming year. 

Technical assistance . Neither the School Improvement Project nor the 
Local School Development Project have the budgetary flexibility to support 
other than field staff and limited administrative personnel. Both projects, 
however, require the skills of social science methodologists during major 
phases of program implementation. The Documentation Unit will provide 
this expertise on an ongoing basis. Technical assistance support services 
will be available to the projects in the development of documentation and 
needs assessment methodologies, and to the schools as they prepare eval- 
uation procedures for the objectives contained in the school plans. 

ERJC lu 
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Analysis and dissemination . A fourth function of the Documentation 
Unit involves the analysis of the documentary and evaluative data collected, 
and the dissemination of the results of these analyses. Various analyses 
of the data have been and will be conducted by the Unit. Until now, these 
analyses have focused on assessing the effectiveness of project activities 
across participating schools; in the coming year analytic papers in 
selected topic areas will also be prepared. 

The results of the data analyses will be communicated through a variety 
of mechanisms, including in-house working papers, formal evaluation reports, 
journal articles, presentations at professional conferences, and the 
research dissemination forums used by MEDARP. One of the expectations of 
the Ford Foundation in funding the Documentation Unit was that information 
regarding the activities and program designs of the School Improvement and 
Local School Development Projects be widely disseminated. Therefore, in 
addition to the Unit's preparation of oral and written presentations of 
findings, seminars, workshops and conferences for various educational groups 
will be jointly sponsored by the Documentatiqp Unit and MEDARP. 

Documentation Unit Staff 

The Documentation Unit staff includes a Unit Manager, two Documentation 
Associates, a secretary, and several educational consultants. The Unit 
Manager reports to the Director of MEDARP and supervises the development 
and implementation of all programmatic and administrative activities of the 
Unit. The Documentation Associates assist the Unit Manager in the develop- 
ment and implementation of documentation and assessment methodologies, data 
collection and analysis, report preparation, and technical assistance to the 
projects. They also plan the activities of educational consultants, who 



n 



8 



assist in data collection and analysis and report preparation. The Unit 
secretary is responsible for all clerical duties and assists the manager 
with various administrative tasks. 
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DOCUMENTATION UNIT METHODOLOGIES AND ACTIVITIES 



In this section, the methodologies and activities to be applied by the 
Documentation Unit, in conducting the documentation, forma ive evaluation, 
technical < ^stance, and analysis and dissemination functions of the Unit 
are described. A calendar of Unit activities between November 1979 and 
August 1981, and the first and second funding period budgets, are also 
included at the end of this section. % 

Documentation Methodology 
A primary objective of the Unit is the documentation of the School 
Improvement and Local School Development Projects. The documentation 
function of the Unit is defined as the establishment and maintenance of 
an ongoing record of the development of the two projects. It is hoped 
that .maintaining such a record will assist the projects, as well as 
future school improvement efforts, in understanding the process of change 
at the school building and system levels. 

c Over the coursfe of the first funding period (November 1979 - August 
1980), documentation efforts focused primarily on project activities at the 
school building level. Emphasfs?was placed on recording SIP and LSDP fieTfl 
staff (school liaisons, District Consultants and Coordinators) activities 
ip participating schools, and the reaction and involvement of school adminis- 
trators, staff, and parents in these activities. This jwas accomplished 
through the collection of field staff logs and progress reports, minutes of 
school' planning t^am/cormri ttqe meetings, notes „op meetings of project 

i 

administrators with field staff and with participating principals, and the 
results of an annual interim project assessment. 
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Durinc Lhe second funding period (September 1980 - August 1981), in 
addition to the documentation of project activities at the school level 
as described above, documentation efforts will be expanded to include 
Community School District involvement in the projects is well as the 
involvement of city-wide constituency groups. Interviews will be conduc- 
ted with participating District Superintendents and local School Board 
members to ascertain their perceptions and degree of involvement in the 
projects. Members of the advisory councils to the projects will also 
be interviewed. 

Documentary information also includes school building and pupil demo- 
graphic data and basic skills test results. Project descriptions and 
proposals, significant memoranda, and various interim reports also serve 
as sources of documentation. 
Schools Included in the Documentation Process 

Because of the inherent difficulties in maintaining a comprehensive 
record of project activities in al 1 of the School Improvement and Local 
School Development schools, a Sc..u . participating schools will be 
selected from each of the projects for documentation purposes. A sample 
of two schools actively involved in the project from each of the five 
participating districts (ten schools) will be selected from among the 
LSDP schools. The nine public and one of the three non-public elementary 
schools presently participating in SIP/ will comprise the ten-school SIP 
sample included in the intensive documentation effort- This sample of 
twenty schools will allow for a concentrated, focused documentation 
effort wnich will be comprehensive, substantive and will provide continuity 
in describing project development and progress at the school level. 

ERIC li 
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Documentation Data Collection Activities "' 

Daily logs and monthly progress reports . The consultant and Coor- 
dinator assigned to each of the five school districts participating in * 
the Local School Development Project are responsible for preparing monthly 
progress reports. These reports summarize the activities conducted by 
the consultant and coordinator as well as the accomplishments of the 
Planning Teams at each of the participating district schools. They also 
include a list of objectives the consultant and coordinator have planned 
for the coming month. 

Liaisons with the School Improvement Project are expected to maintain 
a daily log. In addition to a brief summary of the day's activities, the 
logs also include the liaison's comments on significant events or conver- 
sations which occurred that day. The logs are very useful information 
sources because they provide a daily record of project activities as well 
as an analytic interpretation of developing relationships and trends of 
opinion and reaction in the school. While providing the project adminis- 
trators and the Documentation Unit with an updated account of activities 
in participating schools, the logs and progress reports have also been 
useful to the Unit in developing valid project assessment survey instru- 
* ments. 

The Documentation Unit has been responsible for maintaining a chrono- 
logical file of Monthly Progress Reports (LSDP) by district, and Daily 
Logs (SIP) by school, over the duration of both projects. The first 
Progress Reports are available for January, 1980; the first Daily Logs 
were submitted in October, 1979. 

O 1 r- 
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Planning team/ improvement committee minutes . Each School Improvement 
Committee is responsible for the preparation of formal agendas and minutes 
for all committee meetings. Ihe agendas and minutes serve to document the 
activities and decisions of the committee and are distributed school-wide 
to inform staff and parents of the progress ana direction of the committee. 
School liaisons submit their committee agendas and minutes to the central 
project office. Each liaison also submits a written, subjective analysis 
of what occurred at the meeting. 

Local School Delelopment Planning Teams are not required to maintain 
or submit minutes of their meetings. An attempt will be mdde to 
encourage Planning Teams which are part of the ten-school project sample 
selected by the Documentation Unit, to submit agendas and minutes for all 
meetings. 

The Documentation Unit will maintain a file of agendas and minutes of 
all School Improvement Committee and Piannihy Team meetings, arranged by 
school. Included with the School Improvement Committee minutes are the 
liaison comments regarding the meeting. This information is available 
for all committee meetings which have been conducted in the schools 
participating in|the School Improvement Project. 

Notes of project staff and principal meetings . Both the Local School 
Development and School Improvement Projects hold regular meetings of 
project administrators with the principals of participating schools. 
School Improvement Project principals meet monthly as a group. During 
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1979-80, meetings of the Local School Development principals were held for 
the purpose of training; regularly scheduled district-.wide meetings of pro- 
ject principals are to be initiated in the fall, 1980. Weekly and/or bi- 
weekly meetings also occur between^ the project administrators and their 
respective field staffs. 

Notes are kept at principal and field staff meetings; these meetings are 
very useful to documentation efforts because discussions usually revolve 
around timely issues of project implementation. These notes are also 
referred to by the Unit during preparation of project assessment instruments 
to ensure that all T'gnificant program concerns which developed over the 
course of the year are addressed in annual evaluation activities. 

Interim project assessments . The Documentation Unit periodically 
collects survey information from project staff and administrators, staff and 
parents in participating schools. The purpose of this activity is to 
systematically document the progress of these schools following completion 
of major phases of the projects. Project phases, which may differ in 
sequence and emphasis between the two projects, can be generally identified 
as*: 1) school selection and nroject introduction; 2) school needs assessment; 
3) planning team/improvement committee formation; 4) team/committee planning 
and development of the comprehensive plan; and 5) plan implementation. 

The first interim assessment effort took place during May-June 1980. 
The Documentation Unit conducted interviews with project administrators, 
field staff and principals, and distributed questionnaires to planning 
teams/cormnttee members and to school staff and parents not on the planning 
teamS/conmittees. Based largely on information contained in the daily 
logs and the monthly progress reports, these survey instruments are 

ERIC J ' 
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designed to record the experiences and perceptions of project staff and 
the school community as schools move through the "change process". 

To further document project activities and the change process in 
individual schools, a record of the results of the content analysis of 
interview and questionnaire responses are maintained by school. This 
data augments the individual school information available through the 
logs and progress reports. The interim project assessment results can 
also be used to revise the overall design of the projects where neces- 
sary, and to give the project administrators a clearer sense of the 
progress of the projects following the completion of each year of 
program implementation. Further discussion of the interim project 
assessment activities of the Documentation Unit is provided in this 
section under "Formative Evaluation MethQdology . " 

School building and pupil data . Various types of demographic and 
test score data are being cor.piled by the Documentation Unit for the 
project schools. This data will provide a longitudinal perspective on 
the schools both for the purposes of historical information and the 
identification of trends occurring in these variables over the years 
prior to and during participation in the projects. Small sample sizes 
(it should be noted that the n or "number of subjects" in this case is 
the number of schools) make elaborate statistical analyses such as 
multiple regression inappropriate. However, these variables offer useful 
information in examining alternative hypotheses regarding project effects. 



15 



Drawing upon information sources at the central Board of Education, 
the following school building, pupil, and test score data are being 
collected, starting with the 1974-75 school year: 

School Bui lding Data 

1. Building capacity is defined by the Board of Education as the 
ideal number of pupils who can be physically accommodated in 
the school given its education program. Building capacity is 
largely determined by the restrictions on class size outlined 
in the Teachers' Contract and the use of space in the building 
by other than Board of Education programs, thereby reducing 
the number of available classrooms. 

2. Percent building utilization is the measure of the usage of a 
school building in relation to its rated capacity. 



3. The official school register is based on the number of pupils 
enrolled in the school on the last day in October in a given 
year. 

4. Average daily register is the sum of the daily registers for 
the school year divided by the number of days in the school 
year. Since the enrollment at a school will vary continuously 
with the admissions and departures of pupils during the year, 
the average daily register tends to be a mors reliable 
measure of a school's pupil population than is the official 
October 31 register. 

5. Attendance is the mean pupil attendance rate for the school 
year. Percent attendance is calculated by dividing the average 
daily register into the average daily attendance. 

Pupil Characteristics 

1. Ethnic composition is the total number and proportion of pupils 
within the school falling into the following ethnic categories: 
Black, Oriental, Hispanic, and Other (includes pupils not 
classified under any of the previous categories). 

2 . Socio-e c onomic status variables include - 

a. norm aid to families with dependent children (AFDC)- the 
number of children of an age appropriate to the grade span 
of the school whose families live in the attendance 
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zone of the school and receive assistance under the Hew York 
City AFDC Program. (The Board of Education reports a 95% 
success rate in the matching of addresses of families 
receiving AFDC assistance in a particular school attendance 
zone and the addresses of children on the school's register.); 

b. free lunch eligibles - the number of children in the school 
who qualify for free school lunches under U.S. Department of 
Agriculture regulations due to low family income; and 

c. percent low-income children - a measure of the percentage of 
children from low-income families attending the school 
(based on the number o^ children in the school whose families 
receive AFDC assistance (weighted 60%) plus the number of 
pupils eligible for the Free Lunch Program (weighted 40%)) . 

Achievement Test Data 

Based on the results of the annual city-wide reading and math achieve- 
ment tests, the following information will be collected for each of 
the schools participating in the projects: 

1. tne school's rank in the city-wide ranking of elementary and 
intermediate schools based on the percentage of pupils reading 
at or above grade level; 

2. the school's rank in its Community School District ranking of 
district elementary and intermediate schools based on the 
percentage of pupils reading at or above grade level; 

3. the percent of pupils for the entire school scoring at or above 
grade level, one year or less below grade level, or.e to two 
years bolow grade level, two or more years below grade level; 
and 

4. the mean and median grade equivalent scores for each grade 
level . 

Project documents . For the purposes of documenting the initial concep- 
tualization, implementation, and revision of program designs, significant 
project documents are collected by the Unit. These documents include: 
1) the original project descriptions and proposals prepared by the Board 
of Education and the New York Urban Coalition; 2) project memoranda 
describing or informing staff and participants of varioui project 
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procedures, activities, requirements, etc. (e.g., school selection proce- 
dures, comprehensive/improvement plan formats, training workshops); and 
3) annual and semi-annual progress reports prepared by the project 
administrators and field staff. 

Copies of the final written Comprehensive School Plans (LSDP) and 
School Improvement Plans (SIP) developed at each of the 20 sample schools 
will also be maintained by the Unit. These plans are quite similar to 
traditional Federal grant proposals in format, although they are not as 
elaborate and do not contain as much narrative. While there is variation 
between the two projects, the components to be included in the plans are 
a statement of need, clearly defined goals and objectives, proposed 
solutions and activities, and an evaluation design. 

Discussion s with school district and city-wide constituency represen- 

tatives . During the 1980-81 school year the Documentation Unit plans to 

conduct periodic interviews with a sample of superintende?,ts and School 

3 

Board members ot participating Community School Districts, and representa- 
tives of the various city-wide organizations and agencies which comprise 
the advisory councils to the Local School Development and School Improve- 
ment Projects (see Appendix A for memberships). The purpose of these 
interviews is to document the intensity and type of involvment these 
individuals and groups are having in the projects, as well as their 
perceptions of the progress and success of the projects. These individuals 
should provide interesting perspectives on the projects as they interpret 
project goals and activities from their own particular vantage points. 



J As of June, 1980 five districts were involved in SIP and nine in LSDP. 
Two districts are participating in both projects. 
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Formative Evaluation Methodology 
The formative evaluation activities of the Documentation Unit provide 
project staff, school participants, and program sponsors with useful feed- 
back regarding project effectiveness. # This information will be used to 
modify project activities and program design. Formative evaluation occurs 
both formally and informally throughout the course of the School Improve- 
ment and the Local School Development D rojects. Annual interim project 
assessments provide a more formal vehicle; consultation at field staff 
project meetings ns the primary means of informal feedback. 

Evaluation Activities 

Attendance and feedback at project field staff meetings . Documentation 
Unit staff attend the weekly or bi-weekly meetings of SIP and LSDP field 
staff. At these meetings, feedback on the daily logs (SIP) and monthly 
progress reports (LSDP) is provided. This ongoing interaction of the 
Documentation Unit and field staff ensures that the consultation provided 
by the Unit will be timely as well as maximally useful and relevant to 
project conrerns. This interaction will also encourage greater trust 
between the Unit and the projects, and encourage field staff to be analytic 
in their reflections on activities occurring in the schools. 

Interim project assessments . The Documentation Unit will conduct 
arnual assessments of project activities in the sample of participating 
schools during May-June 1980 and 1981. These assessments will serve to 
further systematize the Unit's documentation efforts, and will provide 
the project administrators and field staff with formative evaluation data. 

I 
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The first interim assessment in 1980 included the following activities 



Review of program objectives - The progress of the projects in 
accomplishing program objectives and implementing activities as 
presents in the original project proposals and descriptions was 
reviewed. This information was useful to project administrators 
in considering the program design as they planned for those 
schools continuing in the projects in September, 1980 as well ?.s 
new schools joining the projects in the fall. 

Collection and analysis of survey information - Structured interviews 
and questionnaires dealing comprehensively with the major phases of 
both projects have been prepared by the Documentation Unit. These 
instruments survey project staff and school participant perceptions, 
attitudes and opinions regarding all aspects of the projects as they 
were implemented over the year. 

Interviews were conducted with project administrators (SIP Director 
and Manager; LSDP Managers), field staff (SIP Liaisons; LSDP District 
Consultants and Coordinators), and with the twenty principals of the 
schools included in the documentation sample (SIP - 9 public and 1 
non-public elementary schools; LSDP - 10 public elementary and inter- 
mediate schools, two from each participating district). Question- 
naires were distributed to the members of the Planning Teams (LSDP) 
and School Improvement Committees (SIP) in the sample schools. Staff 
and parents not on the planning teams and improvement committees also 
received a short questionnaire. 

The second interim assessment will be conducted in the spring, 1981; 

it will include some of the evaluation activities of the first interim 

assessment and a number of additional program assessment components: 

Review of program objectives - The progress of the projects in 
achieving and implementing objectives and activities as described 
in their second year funding proposals will be examined. 

Collection and analyses of survey information - Survey data will be 
collected in the documentation sample schools as during the first 
interim assessment. Additionally, however, interviews will be 
conducted with a parent and staff member of each of the planning 
teams/committees in these schools. A number of newly participating 
project schools will also be included in this evaluation activity 
to document and assess the effects of modification in the program 
designs and activities made following the first year of implementation 
of the projects. 
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Analysis of test score data - To examine the effect the projects have 
had on pupil performance in the basic skills areas, reading and math 
achievement test scores for several years prior to and following 
implementation of project activities, will be reviewed for each 
participating school. Mean and median grade equivalent scores and 
the percentage of students scoring in various achievement categories 
(i.e., at and above grade level, one month to a year below*grade level, 
one to two years be<ow grade level, and more than two years below 
grade level) by grade and by building, will be longitudinally reviewed 
for each school. The purpose of these analyses will be to identify 
trends in the performance of pupils in the basic skills areas rather 
than significance testing of pre/post project implementation test 
score differences. 

Significance testing of reading and math achievement test scores will 
be performed for each of the documentation sample schools. Pupil 
scores will be used to compare grade and building results on the 
city-wide achievement tests administered in the spring, 1979, 1980 and 
1981. A time series design will be used to conduct these analyses. 
District schools with comparable pupil populations not participating 
in the projects may be selected to serve as control groups. 

Review of the success of the written school plans - Using the results 
of the evaluation activities included as a component of each objective 
of the comprehensive school plans/improvement plans, the Unit will act 
as an ■■outside" evaluator in assessing the first year of implementation 
of the activities outlined in the plans of the documentation sample 
schools for their use in planning project activities for the coming 
year. Only the documentation sample schools will be included in this 
activity; however, if this assessment technique proves to be particu- 
larly effective, the Unit will assist the project administrators in 
offering this "outside 11 evaluation service to all participating schools. 

This evaluation activity will be conducted with eligible School 
Improvement Project schools in June 1981 following the first year of 
implementation of school plans. Local School Development Project 
schools will be included in this activity for the first time in 
June 1982, as comprehensive school plans will be completed in June 1981 
and first implemented over the 1981-82 school year. 

Analysis of pre/post school assessment measure s - Evaluation profiles 
have been prepared by the Documentation Unit for the ten School 
Improvement Project sample schools based on the needs assessment data 
(other than test scores) collected in the fall, 1979. This data will 
serve as the pre-project measure of each of the schools on the variables 
making up the five school effectiveness factors discussed earlier. 
"Post measures" of these variables will be collected in the spring, 1981 
at the schools. A comparison of pre/post measures will be made by 
school to ascertain changes in any of these school characteristics 
following two years of school participation in the project. 
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Needs assessments will be conducted in the Local School Development 
schools in the fall, 1980. Evaluation profiles will be compiled for 
the ten-school documentation sample. "Post measures" and pre/post 
comparisons will not be undertaken by the Documentation Unit until 
spring, 1982. 

These school re-assessment activities will provide participating 
schools and project administrators with evaluative information 
regarding the effectiveness of project activities in addressing 
the school needs identified two years previously, as well as a 
current indication of emerging school needs. This knowledge will 
assist schools in the planning of project activities for the 
coming school year. 

Analysis of the information collected during these interim assessments 
will be shared with the project administrators and participating schools 
on a timely basis through concise statistical summaries and internal 
memoranda. This data will be made available to the projects within the 
limits of maintaining the anonymity and confidentiality of individual 
respondents. 

The Documentation Unit will prepare narrative reports summarizing 
the results of the annual project evaluations. These reports will be 
shared with the administrators and staff of the School Improvement and 
Local School Development Projects, the central Board of Education, and 
with funding sponsors. The reports will eventually be made available for 
general dissemination. Again, participant and school anonymity and 
confidentiality will be maintained throughout the reports. (See Appendix 
B for further comments regarding confidentiality of data.) 



The Documentation Unit plans to conduct various analyses of the 
documentary and evaluative data which have been collected. These analyses 
will include the annual assessments of the interview and questionnaire 
data and test score information as well as the preparation of a series 
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of analytic papers in selected topic areas. These papers will draw largely 
on the documentary data collected and will be more descriptive and qualita- 
tive in format than the annual evaluation reports prepared by the Unit.". 

Tentatively, the topics of the analytic papers will include an exam- 
ination of the two models for school site planning implemented by the 
School Improvement and Local School Development Projects. This report will 
focus on the planning processes, the phases and components of the program 
designs, and the organizational conditions necessary for successful 
implementation of such projects. A second paper under consideration is an 
analysis of the aggregated results of the need assessments conducted in the 
ten School Improvement Project schools. These schools comprise a represen- 
tative cross section of the New York City public elementary schools and 
such an analysis will provide an indication of the most pressing concerns 
and issues of the city's elementary schools. A third report will focus 
more closely on the variation in program implementation among the five 
Community School Pistricts participating in the Local School Development 
Project. The LSDP process of school self-improvement is contingent upon 
District Office support for and eventual institutionalization of the 
capacity for school-based planning; therefore, an analysis of this process 
as it has evolved in the participating districts will be both theoretically 
and practically useful. 
Written and Oral Presentation of Finding s 

Written presentation of the results of the Documentation Unit data analyses 
will include the preparation of 1) concise statistical summaries of evaluative 
data for th^ project administrators and, in some cases, for participating schools, 
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2) annual project evaluation reports for the State Education Department, 

3) yearly narrative reports presenting tne results of the Unit's interim 
project assessment activities, and 4) a series of analytic papers. The 
majority of these reports will be available for general dissemination, 
and will also be drawn upon by the Unit in writing articles for wider 
communication of findings in professional publications. 

Oral presentation of the results of the Unit's analyses will occur 
at professional conferences and through a series of workshops/seminars to 
be conducted with the Metropolitan Educational Development and Research 
Project (MEDARP). These workshops/seminars will be tailored to the 
interests of a variety of educational groups including parents, teachers, 
school administrators, and educational researchers. 

Tec»..' * ca1 Assistance 

The Documentation Unit offers technical assistance to the School 

Improvement and Local School Development Projects in a variety of areas, 

particularly in the development of documentation, needs assessment and 
evaluation methodologies. The availability of supportive services from the 

Documentation Unit serves to "repay" the projects for their cooperation 

with the various data and information collection activities of the Unit. 

Activities . , / 

Documen ta tion methodologies . Tne Documentation Unit has provided, and 
will continue to provide, guidance to project administrators in the development 
and modification of sources of program documentation, such as the notes kept 
by field staff (liaison daily logs, District Consultant/Coordinator monthly 
progress reports) and minutes of Planning Team/School Improvement Committee 
meetings. 9 J 
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N eeds assessment methodology . The Unit played a major roit in the 
development of the School Improvement Project needs assessment method- 
ology through the design of data collection instruments, data analysis 
procedures, and the needs assessment report format. 

The Unit developed survey instruments which operational i zed the five 
school effectiveness factors (see Appendix C for a listing of the 
variables making up each of the factors). The survey instruments included 
a'teacher questionnaire, two forms of a teacher interview schedule, and 
interview forms for the principal, assistant principal, special program 
teachers, auxiliaty staff, and paraprofessionals. The Building and 
Grounds Observational Assessment Instrument was also developed to 
periodically evaluate the safety, maintenance, and use of various interior 
and exterior school spaces when assessing the schools along the school 
climate factor. 

The first editions of these instruments were used by the School 
improvement Project to conduct needs assessments in participating schools 
in the fall, 1979. Based on the results of this first administration, 
some of the instruments have been revised by the Documentation Unit. 
These "second editions" are being used by the School Improvement Project 
to conduct needs assessments in srhools scheduled to begin participation 
in the project in the fall, 1980. 

The Documentation Unit has also prepared two new questionnaires to 
replace the Teacher Questionnaire. The original Teacher Questionnaire 
served only to supplement information gathered in the interviews with 
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instructional staff. The new questionnaires (one for classroom teachers 
and one for special progru.n teachers), however, closely parallel the 
items included on the instructional staff interview forms. These new 
questionnaires will be used by the project to shorten the time required to 
complete the data collection phase of the needs assessment; half of the 
instructional staff will receive personal interviews and the other half 
will be asked to complete the new questionnaire or "self-administering 
instructional steiff interview form." These self-administering interview 
forms are among the first of a number of survey instruments which will 
comprise a school self-assessment package being developed by the Unit. 

The Unit also assisted liaisons in the gathering of school building, 
pupil and test score data available from central Board sources. This data, 
was collected for each school for the five year period prior to participa- 
tion in the project and was included in the needs assessment report. 

To assist Liaisons in the interpretation of the interview data, 
content analysis procedures which were used to quantify int* "iew responses 
were developed by the Documentation Unit. The process of establishing the 
response categories for the individual interview itc^s also facilitated the 
revision and refinement of the original needs assessment interview schedules 
and the preparation of the self-administering versions of these instruments. 

A needs assessment report format was outlined by .the Documentation Unit 
to guide the liaisions in the preparation of these reports. The reports 
were narrative in design and consisted of four sections: School Description, 
Methods, Results (presented by effectiveness factor) , and Discussion and 
Conclusions. 



The Unit is preparing a revision of this report format. The new report 
will consist largely of statistical summaries of the assessment data, 
followed by brief interpretations of the results by the liaisons. The 
revision of the report formr* was undertaken to shorten the length of time 
necessary to prepare the report, as well as to increase the amount of 
"objective 11 data presentation and reduce the "subjective" or interpretive 
content of the reports. The new report format has been piloted in one 
school and will be edopted as the required reporting style for the project 
next year. 

The same assistance and support (instrument design, analysis procedures, 
report format) is being offered to the Local School Development Project as 
project staff pi spare and implement their school needs assessments in the 
fall, 1980. Thus far, the Unit has assisted the project in the development 
of the School Profile component of the needs assessment. The LSDP School 
Profiles are longitudinal summaries of various pupil, school building, and 
test score data prepared for each school . 

Evaluation methodology . In addition to assisting in the original 
design of the SIP improvement plan format, the Documentation Unit will 
review and/or develop evaluation strategies for objectives and activities 
included in each school plan. Again, the Unit will offer the same 
assistance tj the Local School Development Project as schools prepare 
their written comprehensive plans over the 1980-1981 school year. 
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Summary of Documentation Unit Activities 

Various documentation, formative evaluation, technical assistance, and 
data analysis activities of the Documentation Unit have been initiated over 
the first funding period (November 1979 - August 1980} of this project. 
The information collected by the Unit for the purpose of documentation of 
the School Improvement and Local School Development Projects has included 
field notes of project staff, notes of project staff and principal meetings, 
minutes of planning team/improvement committee sessions, school building 
and pupil statistical dat&, achievement test scores, and relevant project 
documents. 

The first interim assessment of both projects was conducted 1 
in the spring, 1980, The Unit's analyses of this data resulted in the 
preparation of two preliminary analysis reports which were presented to 
the project administrators in July, two narrative interim project eval- 
uation reports Completed by early fall, and submission of a Title IV-C 
State Education Department Annual Evaluation Report for the School 
Improvement Project by August 31, Staff of the Documentation Unit have "\ 
also provided formative evaluation consultative services on an ongoing 
basis at project staff meetings. 

Technical assistance in terms of various project documentation 
strategies was provided to both projects by the Documentation Unit over 
the first .unding period. Additional technical assistance offered to the 
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The School Improvement Project: First Annual Assessment (August 1980) 
The Local School Developement Project: First Annual Assessment 
(Octuoer 1980). 
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School Improvement Project included support in the development of the 
school needs assessment methodology and the evaluation components of the 
School Improvement Plans. 

Over the second funding period (September 1980-August 1981) a number 
of new activities are being implemented by the Unit. Documentation 
activities will continue as implemented during the first funding period 
and will also include the collection of interview data from school district 
and city-wide constituency representatives involved in the projects. 
Interim project assessment activities will be expanded to include analysis 
of achievement test score data, and evaluation of improvement plan 
activities and comparison of pre/post project implementation school 
assessment data for the School Improvement documentation sample schools. 
Project assessment reports prepared by the Unit in the summer, 1981 will 
also include a Title I/PSEN State Education Department Annual Evaluation 
Report for the Local School Development Project. 

Technical assistance will be offered to the Local ^School Development 
Project in the development of school needs assessment methodologies and 
the evaluation activities to be included in the Comprehensive School Plans. 
Finally, the Unit will prepare a number of analytic reports drawing upon 
the documentary data collected, and will initiate the first in a series of 
workshops/seminars "presenting the findings of the Unit's activities. 
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Schedule of Activities 

Documentation Data Collection Activities 

1. Staff field notes 

Daily logs (SIP) 10/79 - ongoing 

Monthly progress reports (LSDP) 1/80 - ongoing 

2. Minutes of improvement committee/planning team meetings 

Improvement committees (SIP) 2/80 - ongoing 

Planning teams (LSDP) 2/80 - ongoing 

3. Notes of project staff and principal meetings 

SIP staff meetings 1/80 - ongoing 

SIP principal meetings 1/80 - ongoing 

LSDP staff meetings 2/80 - ongoing 

LSDP principal meetings 2/80 - ongoing 

4. Interim project assessments (interviews , questionnaires) May,- June annually 

5. School building and pupil statistical data, June annually 
and achievement test scores 

f 

6. Project documents Ongoing 

7. Interviews with school district and city-wide Mid-school year 
constituency representatives 1980-1981 

Formative Evaluation Activities 

1. Attendance and feedback at project field staff meetings 

' SIP 1/80 - ongoing 

LSDP 2/80 - ongoing 

2. Interim project assessments May - June annually 
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Technical Assistance Activities 



1. Documentation methodologies 

2. Needs assessment methodologies 

SIP 
LSDP* 

3. Development of school plan evaluation components 

School Improvement Plans (SIP) 
Comprehensive School Plans (LSDP) 
Data Analysis/Reports/Dissemination of Results 
1. Preliminary Data Analysis Report of Interim Project 
Assessment Activities prepared for the project 
administrators 

K 2. State Education Department Annual Evaluation Report 
Title IV-C 
Title I/PSEN (LSDP) 

3. Narrative Interim Project Evaluation Report 

4. Analytic Papers 

Examination of the SIP and LSDP Models for 

School Site Planning 
An Assessment of NYC Schools Utilizing 

Edmonds 'School Effectiveness Factors 
District Implementation of the LSDP 

Planning Process 

5. MEDARP Documentation Unit Workshops/Seminars 



Ongoing 

8/79 - ongoing 
9/80 - ongoing 

6/80 - ongoing 
6/81 - orwjoing 

July annually 



8/80 annually 
8/81 annually 
September annually 

November- February , 
1980-81 



1/81 - ongoing 



Documentation Unit Budget for the 
First Funding Period (September 1980-August 1981) 



Personal Services 
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Total Salaries 
Total Benefits 



$46,646 
9,039 



Other than Personal Services 



Office Supplies 

Office Furniture and Equipment 
Postage 

Communications 
Out-of-Town Travel 
Consultant Services 
Special Services 



1,047 
2,256 
150 
528 
233 
7,316 
41 



OTPS Total 
PS Total 
Indirect Costs 

Grand Total 



3, 



371 

€85 
,878 



$71,134 
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Documentation Unit Budget for the 
Second Funding Period (September 1980-August 1981) 



Personal Services 

Total Salaries $93,736 
Total Benefits 28,486 

Other than Personal Services 



Office Supplies 600 

Printing Supplies 3,000 

Office Furniture and Equipment 5,000 

Postage 500 

Communications 100 

Office Services 1 ,000 

Local Travel 1 ,000 

Out-of-Town Travel 1 ,000 

Consultant Services 18,000 

Special Services 1 ,500 



OTPS Total 31 ,700 

PS Total 122,222 
Balance Unallocated 119,834 



Grand Total* $273,756 



*The Grand Total does not reflect Indirect Costs because the Ford Foundation 
has waived these costs over the second funding period. 
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Appendix A: Composition of Project Advisory Councils 



Council on Local School Development (LSDP) 

- New York Urban Coalition 

- Alliance of Business, Labor and Education 

- United Parents Association 

- New York City Board of Education 

- New York Community School Board Association 

- New York Foundation 

- New York State Education Department 

- Council on Children and Families 

- New York City Association of Superintendents 

- Council of Supervisors and Administrators 

- United Federation of Teachers 

- Public Education Association 

Title IV-C Advisory Committee to the School Improvement Project (SIP) 

- New York City Board of Education 

- Office of Catholic Education Diocese of Brooklyn 

- United Parents Association 

- New York City Association of Superintendents 

- New York State Education Department 

- Council of Supervisors and Administrators 

- United Federation of Teachers 

- Public Education Association 
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Appendix B: Availability of Data - 

An overriding concern of the Documentation Unit must be assurance of the 
confidentiality and anonymity of all "raw data" and information sources. That 
is, every attempt is being made to guarantee that no district, school, or 
individual is identified in any manner either in tiles maintained by the 
Documentation Unit or in written or oral reports prepared by the Unit or by 
other individuals or organizations drawing on the data collected by the Unit. 

As a further means of increasing information security, access to the data 
cannot be granted prior to the preparation of a formal research proposal by the 
individual or organization seeking to use the data. The proposal would be 
reviewed and approved by the Directors of the School Improvement Project and/or 
Local School Development Project and the Proposal Review Committee of the Office 
of Educational Evaluation. The proposal would state the uses to be made of 
the data as well as a knowledge of and willingness to comply with all needs 
for confidentiality and anonymity. 

l\ is very clear, based on the type of information that has already been 
collected by the Documentation Unit, that much of the data, if used improperly, 
could be highly damaging to individuals and organizations. A quote taken out 
of context, the insensitive discussion of the needs or problems of a particular 
school, or the careless interpretation of the documented actions or reactions 
of individuals or school communi ties, could destroy the sense of integrity and 
trust which project administrators ansL£ield staff have worked so hard to 
develop between themselves and the schools. 
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Appendix C: Five School Effectiveness Factors 



Presented below are the variables compris, f ig each of the five school 
factors hypothesized by Edmonds (1979) as characterizing more effective schools. 
The Documentation Unit assisted the School Improvement Project in operation- 
alizing the five factors and developed instruments to be used by the School 
Improvement Project in assessing the strengths and weaknesses of participating 
schools in each of the factor areas. This operationalization continues to 
serve as the basis for the School Improvement Project's school needs assessment 
methodology. 

Factor A - Administrative Style 

I. Principal as Instructional Leader 

- Plays a major role in the establishment of schoolwide instructional 
goals, instructional practices, and curriculum. 

- Establishment of formal procedures for staff interaction in 
curricular and instructional matters. 

- Provides ongoing in-service training and instructional supervision. 

• Sees to the coordination of instructional activities within and 
between grades and programs. 

i 

II. Principal as Administrative Head or Organizational Leader 

- Provides sufficient supplies, instructional materials, supplemental 
resources, school equipment. 

- Formalizes and communicates school policies, procedures, and 
regulations regarding such topics as school safety, discipline, 
attendance, grading, promotion, etc. 

4. - Clearly establishes and communicates the responsibilities and 

duties of staff assignments. 

- Establishes effective (responsive) lines of communication between 
* himself, staff and parents. 

• Provides for staff input in administrative matters. 

Factor B - Instructional Emphasis on Basic Skills (Reading, Mathematics, Writing ) 

I. School -wide 

- Presence of effective school -wide basic skills programs. 

- Presence, acceptance of and adherence to a school-wide written 
curriculum and specific instructional objectives by grade and 
basic skill subject areas. 

- Formalized amount of classroom instructional time set aside for basic 
skills instruction school-wide. 

- Sufficient instructional materials (texts, kits, etc.) in the basic 
q skills subject areas. 
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II. Classroom 

- Type and the amount of group basic skills classroom instruction 
(small group, total group). 

- Individualized instructional planning. 

- Written daily plans. 

- Active student participation in basic skills instruction. 

- Use of commercial reading/math kits. 

Factor C - School CI imate 

I. Physical Environment 

- Orderly school atmosphere 

- Sufficient building security. 

- Maintenance of interior and exterior building spaces in terms of 
repair and cleanliness. 

II. Social Tone 

- Positive staff morale 

- Positive pupil attitude 

- Effective lines of communication between the principal, staff and 
parents. 

- Consensus of school staff and parental opinion regarding educational 
goals. 

Factor D - Ongoing Diagnosis and Assessment of Pupil Progress 

- Instructional use of achievement, diagnostic, and criterion- 
referenced test results. 

- Instructional use of teacher-made tests. 

- Ongoing individualized pupil evaluation and instructional planning. 

- Monitoring of pupil progress based on teacher judgment of pupil 
performance. 

- An effective school -wide record keeping system to facilitate 
teacher awareness of student progress. 

- Regular communication with parents regarding pupil progress. 

Factor E - Teacher Expectations 

- Optimistic expectancies for pupil achievement. 

- Teacher enthusiam - "All children can learn." 

- Encouragement of active student participation in instructional 
activities. 

- Setting of instructional goals and objectives which challenge pupil 
abilities. 

In addition to the questionnaire and interview it*ms related to the five 
factors, the needs assessment survey instruments include open-ended items which 
give respondents an opportunity to express their perceptions of the school's 
strengths, weaknesses and problem areas, as well as suggest potential 
solutions, in any areas not covered by the five factors. 



